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1 Introduction 


The work performed in this period (along with work in preceeding periods) 
is reported in a PhD Dissertation entitled Video Transmission on ATM Net- 
works by Y-.C. Chen, and a paper presented at the International Telecom- 
munication Conference. We give a brief description of the work below, with 
details left to the dissertation and paper which are included with this re- 
port. Also included is a paper on constrained joint source/channel coding 
submitted to the IEEE Journal on Selected Areas of Communications. 

Currently we are continuing with work in both areas reported here. In par- 
ticular, we are examining a number of strategies for error concealment in 
packet video, and extending the vector quantization work. 


2 Video Transmission on ATM Networks 

The emergence of broadband ISDN as the network for the future brings with 
it the promise of integration of all proposed services in a flexible environment. 
In order to achieve this flexibility, asynchronous transfer mode (ATM) has 
been proposed as the transfer technique. This ATM-based B-ISDN network 
will be the carrier for services like HDTV, interactive television, multime- 
dia workstation, and a lot more. As can be seen from these applications 
the proposed network will carry a lot of video information. In the past, 
video coding algorithms have been developed mainly for use over dedicated 
communication links. Although many advantages can be foreseen from this 
new environment, video transmission over ATM networks also present seri- 
ous challenges to network providers and video specialists. These include the 
development of fair and efficient resource allocation schemes, policing func- 
tions, and more importantly, from the point of view of the video specialist, 
the bridging of network- based performance parameters and controls to the 
corresponding entities in video coding. 

During this period we conducted a study on the bridging of network trans- 
mission performance and video coding. In the ideal case one would have have 
real time simulators for both video codec and network. The interactions be- 
tween these two elements could then be studied extensively. However, it 
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would require a huge amount of effort to build a real time simulator. Given 
the fact that there is still a lot of uncertainty about proposed video coding 
algorithms and network protocols, building a simulator that would handle all 
the different scenarios is not feasible. The approach taken in this work is to 
deal with each key component in packet video separately. By doing that, we 
hope to obtain an in-depth understanding of the whole problem and come 
up with suitable solutions. 

The successful transmission of variable bit rate video over ATM networks 
relies on the interaction between the video coding algorithm and the ATM 
networks. Two aspects of networks that determine the efficiency of video 
transmission are the resource allocation algorithm and the congestion control 
algorithm. 

The resource allocation algorithm dictates the cost and blocking probability 
of a connection depending on the traffic’s characteristic. An efficient resource 
allocation scheme increases network utilization and therefore decreases the 
cost of transmission. A promising approach to resource allocation is equiva- 
lent bandwidth allocation [1]. This approach not only describes the required 
bandwidth for different traffic scenarios based on traffic characteristics and 
quality of service(QOS) requirements but is easy to manage as well. 

The congestion control algorithm is a major factor in determining the quality 
of a call. The policing function plays a vital role in monitoring traffic flow and 
thus maintains a well-operated network situation. Unfortunately, because of 
the variety of traffic, it is not an easy task to effectively regulate connection 
to its agreed-upon contract effectively. Of the schemes proposed to date 
the leaky bucket(LB) algorithm comes closest to being effective. We propose 
a dual leaky bucket mechanism based on equivalent bandwidth assignment, 
with the first bucket monitoring the mean bandwidth and the second one 
monitoring the equivalent bandwidth. With such a design, a misbehaved 
connection can be easily detected and network congestion can be prevented 
effectively (if resource allocation is performed appropriately). Also network 
utilization is effective with a good resource allocation scheme which takes 
advantage of multiplexing gain. Other congestion control approaches which 
have effects in video codec design will also be investigated. 

Based on the understanding of the transmitting channel, we propose a com- 
plete set of design principles for video codecs. Closely following the concept 
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of the dual leaky bucket mechanism, a prioritized coding scheme is presented 
and its performance is studied. We also develop some combined approaches 
to smooth the video output flow. This in turn leads to a reduction in the 
requested equivalent bandwidth. Finally, some error control algorithms are 
proposed to combat the effect of cell loss which comes from the nature of 
packet video. 

There are still a lot of issues about B-ISDN left to be clarified. It will require 
extensive efforts in order to clear the confusion among user, service provider, 
and equipment manufacturer and accelerate the pace of implementing B- 
ISDN. This work provides a design approach for video transmission based on 
the understanding and evaluation of current ATM networks. We also hope 
that the results and conclusions presented in this work may contribute to 
create a guideline for the design of packet video codec in the future. 


3 Vector Quantization for Nonstationary Sources 

Introduction 

Vector quantization (VQ) is one of the more popular compression techniques 
to appear in the last twenty years. Numerous compression techniques, which 
incorporate VQ, have been proposed. While the LBG VQ [2] provides ex- 
cellent compression, there are also several drawbacks to the use of the LBG 
quantizers. These include search complexity and memory requirements, espe- 
cially at higher rates, and a mismatch between the codebook and the inputs. 

The latter mainly stems from the fact that the VQ is generally designed for a 
specific rate and a specific class of inputs. When the bandwidth constraints 
and/or the source statistics change this can result in severe degradation in 
the quality of the reconstructed output. 

In order to reduce the search complexity, a number of techniques have been 
proposed which impose structure on the codebook entries. These include 
tree structured VQs, lattice VQs, and classified VQs [3]. However, each 
approach has its own drawbacks. The tree structured and classified VQs do 
nothing about the memory requirements. In fact the tree structured VQs can 
actually excarbate the memory requirements problem. The lattice quantizers 
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can avoid both search and memory problems, however, these quantizers lack 
the pattern matching ability of the LBG VQ. Residual quantizers (RQs), 
also known as multiple-stage VQs, have been introduced to reduce both the 
computation and memory requirements [4, 5, 6]. 

All of these methods assume that the code book, large or small, accurately 
reflects the input image statistics. However this is not always the case, espe- 
cially when coding a nonstationary source such as a video sequence. Various 
approaches which use some sort of codebook adaptation have been proposed 
in which elements of the codebook are replaced as coding proceeds [7, 8, 9, 10]. 
These approaches are generally of a forward adaptive nature in that the up- 
date procedure requires the transmission of side information. 

In this work, we propose an adaptive technique for vector quantization of 
images and video sequences. The technique is an extension of the recursively 
indexed scalar quantization (RISQ) algorithm [11]. This approach involves 
the use of a small code book, reducing the computational complexity. The 
code book adapts to the input statistics. We present both forward and 
backward adaptation rules. 


Proposed Quantizer 

While the RISQ algorithm has been quite successful in a number of applica- 
tions, it is not possible to directly extend it to vector quantization as there 
are some fundamental differences between scalar and vector quantizers. The 
input to a scalar quantizer is assumed to be iid. The vector quantizer on the 
other hand can be viewed as a pattern matching algorithm [12]. The input is 
assumed to be one of a number of different patterns. The scalar quantizer is 
used after the redundancy has been removed from the source sequence, while 
the VQ takes advantage of the redundancy in the data. 

With these differences in mind we view the recursively indexed vector quan- 
tizer (RIVQ) as a two stage process. The first stage performs the normal 
pattern matching function, while the second stage recursively quantizes the 
residual if the magnitude of the residual is greater than some prespecified 
threshold. The codebook of the second stage is ordered so that the magni- 
tude of the codebook entries is a nondecreasing function of its index. We 
then choose an index / which will determine the mode in which the RIVQ 
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operates. 

The quantization rule Q is given as follows: 

For a given input value x 0 , we have the following: 

• Quantize xo with the first stage quantizer Q i. 

• If the residual ||xo — Qi( x o)|| is below a specified threshold then Qi( x o) 
is the nearest output level. 

• Otherwise generate X\ = Xq - Qi(xo) and quantize using the second 

stage quantizer Q2. Check if the index J\ of the output is below the 
index I. If so, Q(zo) = Qi(x 0 ) + If not, form x 2 = x 2 — Q(x 1) 

and do the same for the same as for x\. 

This process is repeated until for some time m, the index J m falls below the 
index /, in which case xo will be quantized to 

= < 5 i(a^o) + Q 2(^1) + + Q2(%m)- 

Thus, the RIVQ operates in two modes: it operates in one mode when the 
index J of the quantized input falls below a given index I and another when 
the index J falls above the index I. 


Methods for updating the code book 

In this section, we present two algorithms used to update the first stage 
quantizer. The adaptation algorithms use the fact that with the RIVQ the 
output values are always within a prescribed distance of the inputs. This 
means that the set of output values of the RIVQ can be viewed as an ac- 
curate representation of the inputs and their statistics. In the following we 
first present a backward adaptive algorithm which uses only the outputs for 
adaptation, and an adaptation algorithm which uses some side information 
for adaptation. We have called the algorithm forward adaptive even though 
this algorithm also uses the past outputs for adaptation. 


5 



Backward Adaptive Quantization In this method, we divide the input 
sequence into intervals and use the outputs of quantizer of the previous in- 
terval as a training sequence and the present code book as initial code book 
for the generalized Lloyd algorithm. By clustering all the past quantized out- 
puts, a new code book is generated. This new code book is used to encode 
the next block of input. We note that in the beginning the training sequence 
is small depending on the length of the update interval, but after some time, 
the length of the training sequence increases. The second stage VQ could 
also be updated in a similar fashion. In this method no overhead is needed 
since both the encoder and the decoder generate the same code book. 


Forward Adaptive Quantization In this approach we treat a subset of 
the output set of the previous intervals as our codebook. We use the method 
described in [13] to inform the receiver of which elements of the previous 
outputs form the codebook for the next interval. Suppose an output set, in 
order of first appearance, is {p, a, q, s, /, f,r}, and the desired codebook for 
the interval to be encoded is {a, qr , /, r } , then we would transmit the binary 
stringOllOlOl to the receiver. The Is correspond to the letters in the output 
set which would be elements of the desired codebook. We select the subset 
for the current interval by finding the closest vectors from our collection of 
past outputs to the input vectors of the current set. This means that there 
is an inherent delay of one interval imposed by this approach. The overhead 
required to send the codebook selection is M/N where M is the number of 
vectors in the output set, and N is the interval size. 

Preliminary Simulation Results 

We simulated the proposed technique using the Lena image and the Xray, 
Couple and Girl images from the USC database. The images were divided 
into 4x4 blocks (dimension 16). The initial code books for both stages were 
generated using the USC Girl and the USC Couple images as the training 
set. The threshold was chosen to be 500. The update interval was every 8192 
pixels or 512 vectors. 

We compared the results of the proposed systems with an LBG VQ with 
the test image outside the training set and where the test image was also 


6 



the training image. As expected the proposed approach significantly outper- 
formed the LBG VQ with the mismatched training image. However even for 
the case where the LBG VQ used the same image as both training and test 
image, the performance of the proposed system was very close to the LBG 
VQ. We are currently using the proposed technique to code video sequences 
which we will compare to the MPEG algorithm. 

In all cases that the adaptation is very robust, and the performance close to 
the ideal omniscient case. Therefore the technique could be used in situa- 
tions where the source statistics are unknown or change rapidly. We intend 
to present results further justifying these conclusions by the time of the con- 
ference. 
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Vector Quantization of Non Stationary Sources* 
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Abstract 

Common problems with Vector Quantization (VQ) 
include encoding complexity , memory requirements , 
a mismatch between training and test statistics, and 
overload * errors. These problems become increasingly 
pronounced when dealing with nonstationary sources 
such as a video source. We propose an adaptive vec- 
tor quantization algorithm which uses an extension of 
the recursively indexed scalar quantizer to resolve these 
problems. The use of a recursively indexed VQ re- 
sults in distortion limited outputs which can be used 
in adaptive algorithms. We present a backward adap- 
tive algorithm and another which could be classified as 
forward adaptive . 


1 Introduction 

Vector quantization (VQ) is one of the more popu- 
lar compression techniques to appear in the last twenty 
years. Numerous compression techniques, which in- 
corporate VQ, have been proposed. The basic tech- 
nique is simple. The source sequence is blocked into 
vectors. These vectors are compared to a set of rep- 
resentative vectors called a codebook. The index of 
the vector in the codebook which provides the closest 
match (generally in the sense of the L\ or L 2 norm) 
is then transmitted. The codebook is usually gener- 
ated using a clustering algorithm. The most popular 
algorithm for codebook generation is the generalized 
Lloyd algorithm proposed by Linde, Buzo, and Gray 
[1]. There are several drawbacks to the use of the 
LBG quantizers. These include search complexity and 
memory requirements, especially at higher rates, and a 
mismatch between the codebook and the inputs. The 
latter mainly stems from the fact that the VQ is gen- 
erally designed for a specific rate and a specific class of 
inputs. When the bandwidth constraints and/or the 

•This work was supported by the NASA Goddard Space 
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source statistics change this can result in severe degra- 
dation in the quality of the reconstructed output. 

There have been a number of attempts at overcom- 
ing these problems. In order to reduce the search 
complexity, a number of techniques have been pro- 
posed which impose structure on the codebook en- 
tries. These include tree structured VQs, lattice VQs, 
and classified VQs [2]. However, each approach has 
its own drawbacks. The tree structured and classified 
VQs do nothing about the memory requirements. In 
fact the tree structured VQs can actually excarbate 
the memory requirements problem. The lattice quan- 
tizers can avoid both search and memory problems, 
however, these quantizers remove no redundancy from 
the source. 

Residual quantizers (RQs), also known as multiple- 
stage VQs, have been introduced to reduce both the 
computation and memory requirements. In the RQs, 
each stage uses a small code book to encode the errors 
(residuals) of the preceding stage [3]. Recently, Barnes 
and Frost [4, 5] investigated the use of direct-sum code 
book with RQs to minimize the memory requirements 
of VQs. In their work, the RQ stages are jointly opti- 
mized. They concluded that their new design method 
led to an improvement in performance of the VQs. 

All of these methods assume that the code book, 
large or small, accurately reflects the input image 
statistics. However this is not always the case, es- 
pecially when coding a nonstationary source such as 
a video sequence. To guard against degradations in 
the quality of the reconstruction one could use a large 
“universal” codebook which was trained using vectors 
from a number of statistically different sources [6]. 
Unfortunately, due to the size requirements this can 
excarbate the search and memory requirements. Fur- 
thermore it is a minimax solution, which while provid- 
ing some insurance against severe degradation, does 
not provide the best performance for a given code- 
book size. These problems can be somewhat allevi- 
ated by using only a subset of the universal codebook 
at any given time [7]. The composition of this sub- 



set depends on local statistics and is transmitted to 
the decoder as side information. This avoids the mini- 
max problem while retaining the “universality” of the 
codebook. Other approaches which use some sort of 
codebook adaptation have also been proposed in which 
elements of the codebook are replaced as coding pro- 
ceeds [8, 9, 10, 11]. These approaches are generally of 
a forward adaptive nature in that the update proce- 
dure requires the transmission of side information. 

In this paper, we propose an adaptive technique for 
vector quantization of nonstationary sequences. As 
our application area we use image coding. The tech- 
nique is an extension of the recursively indexed scalar 
quantization algorithm [12]. This approach involves 
the use of a small code book, reducing the compu- 
tational complexity. The code book adapts to the in- 
put statistics. We present both forward and backward 
adaptation rules. 

This paper is organized as follows. First, the basic 
technique involving the estension of the RISQ algo- 
rithm is introduced in section 2. In section 3, methods 
used to update the code book are presented . This is 
followed in section 4 by some preliminary simulation 
results. 


2 The Recursively Indexed Vector 
Quantizer (RIVQ) 

In [12] a Recursively Indexed scalar quantizer 
(RISQ) was presented. The RISQ algorithm is briefly 
described as follows. 

For a given quantizer stepsize A and a positive in- 
teger K, define x/ and x/, as follows: 


x h = x\ + (I< - 1)A 

where [xj is the largest integer not exceeding x. A 
recursively indexed quantizer of size K is a uniform 
quantizer with step size A (the uniform spacing both 
between the thresholds and between the output lev- 
els) and with x/ and x& being its smallest and largest 
output levels (Q defined this way always has 0 as an 
output level). The quantization rule Q is given as fol- 
lows: 

For a given input value x if x falls in the interval 
(x/ -f (A/2),x* - (A/2)), then Q(x) is the nearest 
output level. If x is greater than x* — (A/2), see if 

xi=x - x h e (x/ + (A/2), xh - (A/2)). 


If so, Q(x) = (x*,Q(xi)). 

If not, form X 2 = x — 2x^ and do the same as for 

Xi. 

This process continues until for some m, x m = x — 
mxh falls in (x/(A/2),x/ l - (A/2)), in which case x 
will be quantized into 

Q(x) — (^/i> • * * , x h , Q{x m )) 

If x is smaller than x/ + (A/2), a similar procedure to 
this is used, i.e., x m = x — mx/ is formed so that it 
falls in (xj + (A/2),x^ — (A/2)), and is quantized to 
(x/,x/, • • *,X/,^(x m )). 

In summary, the quantizer operates in two modes: 
it operates in one mode when the input falls in the 
range (x/ + -yjX* — y), and another when the input 
falls outside of the specified range. The distortion per 
sample is always bounded by y . 

It is not possible to directly extend the RISQ to vec- 
tor quantization as there are some fundamental differ- 
ences between scalar and vector quantizers. The input 
to a scalar quantizer is assumed to be lid . The vector 
quantizer on the other hand can be viewed as a pat- 
tern matching algorithm [13]. The input is assumed to 
be one of a number of different patterns. The scalar 
quantizer is used after the redundancy has been re- 
moved from the source sequence, while the VQ takes 
advantage of the redundancy in the data. 

With these differences in mind we view the recur- 
sively indexed vector quantizer (RIVQ) as a two stage 
process. The first stage performs the normal pattern 
matching function, while the second stage recursively 
quantizes the residual if the magnitude of the resid- 
ual is greater than some prespecified threshold. The 
codebook of the second stage is ordered so that the 
magnitude of the codebook entries is a nondecreasing 
function of its index. We then choose an index I which 
will determine the mode in which the RIVQ operates. 

The quantization rule Q is given as follows: 

For a given input value x 0 , we have the following: 

• Quantize x 0 with the first stage quantizer Q\. 

• If the residual ||xq — Qi(xq)|| is below a specified 
threshold then Qi(xo) is the nearest output level. 

• Otherwise generate x\ = x 0 - Qi(x 0 ) and quantize 
using the second stage quantizer Qi. Check if the 
index J\ of the output is below the index I. If so, 

Q{* o) = Qi(x 0 ) + £?2(*l)- 

If not, form 

*2 = *2 - Q(X i) 

and do the same for the same as for x\. 



This process is repeated until for some time m, the 
index J m falls below the index /, in which case x Q will 
be quantized to 

Q(xq) = Qi(x 0 ) + <32(^1) 4 - + Q 2 (xm)‘ 

Thus, the RIVQ operates in two modes: it operates 
in one mode when the index J of the quantized input 
falls below a given index I and another when the index 
J falls above the index I. 


\I\ 

Rate 

bits/pixel 

# inputs 
to stage 2 

# outputs 
from stage 2 

PSNR 

20 

1.64 

3378 

8855 

31.97 

30 

1.34 

3590 

6385 

30.74 

40 

1.17 

3590 

4951 

29.67 


Table 1: Effect of |/| on the backward adaptive RIVQ. 


3 Methods for updating the code book 

In this section, we present two algorithms used to 
update the first stage quantizer. The adaptation al- 
gorithms use the fact that with the RIVQ the output 
values are always within a prescribed distance of the 
inputs. This means that the set of output values of 
the RIVQ can be viewed as an accurate representa- 
tion of the inputs and their statistics. In the following 
we first present a backward adaptive algorithm which 
uses only the outputs for adaptation, and an adapta- 
tion algorithm which uses some side information for 
adaptation. We have called the algorithm forward 
adaptive even though this algorithm also uses the past 
outputs for adaptation. 

3.1 Backward Adaptive Quantization 

In this method, we divide the input sequence into 
intervals and use the outputs of quantizer of the pre- 
vious interval as a training sequence and the present 
code book as initial code book for the generalized 
Lloyd algorithm. By clustering all the past quantized 
outputs, a new code book is generated. This new code 
book is used to encode the next block of input. We 
note that in the beginning the training sequence is 
small depending on the length of the update interval, 
but after some time, the length of the training se- 
quence increases. The second stage VQ could also be 
updated in a similar fashion. In this method no over- 
head is needed since both the encoder and the decoder 
generate the same code book. 

3.2 Forward Adaptive Quantization 

In this approach we treat a subset of the output set 
of the previous intervals as our codebook. We use the 
method described in [7] to inform the receiver of which 
elements of the previous outputs form the codebook 
for the next interval. Suppose an output set, in order 
of first appearance, is {p, a, q , $, /, r}, and the desired 


codebook for the interval to be encoded is {a, < 7 ,/, r}, 
then we would transmit the binary stringOl 10101 to 
the receiver. The Is correspond to the letters in the 
output set which would be elements of the desired 
codebook. We select the subset for the current inter- 
val by finding the closest vectors from our collection 
of past outputs to the input vectors of the current 
set. This means that there is an inherent delay of one 
interval imposed by this approach. The overhead re- 
quired to send the codebook selection is M/N where 
M is the number of vectors in the output set, and N 
is the interval size. 


4 Preliminary Simulation Results 

We simulated the proposed technique by applying it 
to the image compression problem. As our test images 
we used a 256 x 256 section of the Lena image and 
the Xray image from the USC database. There are 
several parameters in the RIVQ that can be adjusted 
to vary the performance. These include the threshold 
for quntization of the residual, the second stage index 
/. We do not as yet have an analytical approach to 
setting these parameters, and we present some initial 
results from empirical evaluations. 

The image is divided into 4x4 blocks, therefore 
the VQs have dimension 16. The initial code books 
for both stages were generated using the USC girl and 
the couple images as the training set. The threshold 
was chosen to be 500. The update interval was every 
8192 pixels or 512 vectors. 

Table 1 shows the results for the backward adap- 
tive approach with different values of the index I. The 
value shown in the table is actually the magnitude of 
the element in the codebook table at index I. The im- 
age used here is the a 256x256 portion of the Lena im- 
age. For reference a codebook designed using the Lena 
sub-image as both the training and test input provides 
a PSNR of 32.77 dB at a rate of 1.25 bits/pixel. 

The difference between this and the ideal case at 
the same rate is around 2.5 dB. There is a distortion- 
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m 

Rate 

# inputs 
to stage 2 

# outputs 
from stage 2 

PSNR 

20 

1.69 

3030 

7342 

32.46 

30 

1.44 

3173 

5312 

31.42 

40 

1.31 

3266 

4280 

30.49 


Table 2: Effect of \I\ on the forward adptive RIVQ. 


m 

Rate 

# inputs 
to stage 2 

# outputs 
from stage 2 

PSNR 

30 

1.30 

3426 

6055 

31.02 

40 

1.15 

3542 

4797 

30.00 


Table 3: Performance of restricted forward adptive 
RIVQ. 


rate tradeoff which is a function of I . As I increases 
the PSNR and the rate both drop off. 

In Table 2 we look at the effect on the performance 
of the forward adaptive scheme when the index value 
I is changed. 

Notice that as |J| becomes larger the number of 
requantizations goes down, as would be expected, but 
the number of inputs with residual magnitudes greater 
than the threshold goes up. The reason for this is that 
as the value of [7| goes up, the reconstruction accuracy 
goes down. This leads to codebooks that are not as 
representative of the input which in turn results in 
larger residuals. 

If instead of allowing the encoder to select a code- 
book from the entire set of past outputs we use only 
the last 256 outputs as the codebook, we remove the 
overhead involved in codebook transmission. The re- 
sults for this case are shown in Table 3. 

Comparing these results to those in Table 2, we see 
that there has been a slight drop in rate accompa- 
nied by a slight drop in PSNR values. If we look at 
the number of requantizations we see that these have 
actually increased, so some of the savings from the 
overhead gets used in the requantization. 

In all cases that the adaptation is very robust, and 
the performance close to the ideal omniscient case. 
Therefore the technique could be used in situations 
where the source statistics are unknown or change 
rapidly. We intend to present results further justifying 
these conclusions by the time of the conference. 
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ABSTRACT 

We examine the design of joint source/channel coders in situations where there 
is residual redundancy at the output of the source coder. We have previously 
shown that this residual redundancy can be used to provide error protection 
without a channel coder. In this paper we extend this approach to conven- 
tional source coder/convolutional coder combinations. We also develop a fam- 
ily of nonbinary encoders which more efficiently use the residual redundancy 
in the source coder output. We show through simulation results that the pro- 
posed systems outperform conventional source-channel coder pairs with gains 
of greater than 10 dB in the reconstruction signal to noise ratio at high prob- 
ability of error. 
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1 Introduction 


One of Shannon’s many fundamental contributions was his result that source coding and 
channel coding can be treated separately without any loss of performance as compared to an 
optimum system [1]. The basic design procedure implied by Shannon’s theorems consists of 
designing a source encoder which changes the source sequence into a series of (approximately) 
independent, equally likely binary digits followed by a channel encoder which accepts binary 
digits and puts them into a form suitable for reliable transmission over the channel [1], One 
aspect of the overall optimum system not addressed by Shannon is any increase in system 
complexity that results from this separation. Massey [2] and Anchetta [3] showed that for 
distortionless transmission of the source under the constraint of linear source and channel 
coders, a significant reduction in complexity with equivalent performance can be achieved 
using a joint source/channel coder. Their scheme also differs from most data compression 
schemes in that the bulk of the system complexity is transferred to the receiver. 

The theorem that provides justification for the separate design of the source coder and the 
channel coder, often called the Information Transmission Theorem [1], assumes that both 
the source encoder/decoder pair and the channel encoder/decoder pair are operating in an 
optimal fashion. Specifically, the source encoder is assumed to present the channel encoder 
for optimal channel coding, and the channel encoder/decoder pair is assumed to reproduce 
the source encoder output at the source decoder input with negligible distortion. Unfor- 
tunately, there are practical situations in which these assumptions are violated - namely, 
when the source encoder output contains redundancy, which occurs when the source encoder 
is suboptimal, and when the source decoder input differs from the source encoder output, 
which is a result of channel errors. These two situations are common occurrences in prac- 
tical communication systems where source and/or channel models are imperfectly known, 
complexity is a serious issue, or significant delay is not tolerable. Various approaches have 
been developed to handle these situations, and they are usually grouped under the general 
heading of joint source/channel coding 

We have attempted to develop a more precise nomenclature by distinguishing three classes 
of coders. One class of coders we designate as joint source channel coders because the source 
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and channel coding operations are truly integrated, and in this class we include the work of 
Anchetta [3] and Massey [2], the work of Dunham and Gray [4], who proved the existence of 
joint source/channel trellis coders for certain fidelity criteria, and the joint source/channel 
coder designs of Ayanoglu and Gray [5]. 

In a second class, denoted as concatenated source/channel coders, we place coders that cas- 
cade known source coders and known channel coders, and allocate the fixed bit rate between 
the source coder and the channel coder to maximize the system performance. Work in this 
class includes that of Modestino and Daut [6] who investigated two dimensional differential 
pulse code modulation (2D-DPCM) for image coding combined with short constraint length 
convolutional codes, Modestino, Daut, and Vickers [7] who investigated 2D discrete cosine 
transform (DCT) of images with convolutional codes, Modestino, Bhaskaran, and Anderson 
[8] who studied tree encoding of images with convolutional coding, Comstock and Gibson [9] 
who considered 2D-DCT coding of images in conjunction with Hamming codes, Moore and 
Gibson [10] who evaluated DPCM speech encoding with self-orthogonal convolutional codes, 
Reininger and Gibson [11] who studied backward adaptive prediction in DPCM speech cod- 
ing along with high rate convolutional codes, and Goodman and Sundberg [12, 13] who 
develop embedded DPCM speech encoding and punctured convolutional codes. 

Constrained joint source/channel coding is our third class of coders and is so named because 
the source coder and/or receiver are modified to account for the presence of a given noisy 
channel. In this class of coder we place source coders which have been (re-)optimized subject 
to a noisy channel constraint, such as the work by Kurtenbach and Wintz [14] on memoryless 
scalar quantization for discrete memoryless channels, Farvardin and Vaishampayan [15] on 
memoryless scalar quantizers and codeword assignments for the binary symmetric channel 
(BSC), Vaishampayan and Farvardin [16] on 2D-DCT image coders for the BSC, Kumazawa 
et al., on Linde, Buzo, Gray (LBG) vector quantization (VQ) of Gaussian sources for the 
BSC, and Chang and Donaldson [17] on the optimization of DPCM systems for noisy channel 
operation. Additional work on assigning binary codewords to quantizer outputs for noisy 
channels can be found in the Rydbeck and Sundberg [18], DeMarca and Jayant [19], and 
Zeger and Gersho [20]. 

Another subset in the class of constrained joint source/channel coders are those that use 
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some knowledge of the source or source coder properties to detect channel errors and com- 
pensate for their effects. In this group we include the work by Steele and Goodman, and 
Steele, Goodman, and McGonegal [21, 22] who detect errors in a speech coder output by 
monitoring the sample-to-sample differences in the reconstructed values and replacing those 
reconstructed values whose differences are too large by the output of a smoothing circuit, 
the work of Ngan and Steele [23], and Pitt, Swanson, and Yuen [24] who use similar ideas 
to motivate the development of a method to recover from errors in an image transmission 
system, the work of Reininger and Gibson [25] who use coefficients from neighboring blocks 
in a 2D-DCT image coding system to detect errors and smooth out their effects, and the 
work of Sayood and Borkenhagen [26, 27] who use redundancy in the coder output to per- 
form sequence estimation. Heilman [28, 29] also suggests using the natural redundancy to 
correct errors in joint source/channel coding, and proposes a rate 1 catastrophic code to aid 
the process. The research described in this work is an extension of the work of Sayood and 
Borkenhagen [26, 27], and generalizes the approach of [21]— [25]. An earlier version of this 
work was presented in [30]. 

In the following section we describe the design criterion that is used for the various ap- 
proaches presented in this paper. This is used in the following section to motivate some 
modification of existing source coder/convolutional coder design. The proposed modifica- 
tions are evaluated using simulations, and the results of the evaluations are used to propose 
a class of non-binary convolutional encoders. Simulation results are presented which show 
that the proposed designs substantially outperform conventional systems (in the assumed 
scenario). 

2 The Design Criterion 

For a discrete memoryless channel (DMC), let the channel input alphabet be denoted by 
A = {a 0 , ax, ... , aM-i , }, and the channel input and output sequences by Y = {y 0 , J/i , • • • , 
VL-i} and Y = {yo,Vi, ■ • • ,2/L-i}, respectively. If A = {A,} is the set of sequences A,- = 
{a ti o, a,,i,..., a; t £,_i }, a^eA, then the optimum receiver (in the sense of maximizing the 
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probability of making a correct decision) maximizes P[C], where 

P[C] = Z p iC\Y]P[Y] 

Ai 

This in turn implies that the optimum receiver maximizes P[C\Y\. When the receiver selects 
the output to be A k , then P[C\Y) = P[Y = A k \Y]. Thus, the optimum receiver selects the 
sequence A k such that 

P[Y = A k \Y) > P[Y = Ai\Y ] Vi 

Noting that 

p(y\y = 

P{Y) 

and for fixed length codes P(Y) is irrelevant to the receiver’s operation, the optimal receiver 
maximizes P(Y\Y)P(Y). If we impose a first order Markov assumption on {y,}, we can 
easily show that [31] 


P(Y\Y)P(Y) = nmte)/%l*-i) (1) 

This result addresses the situation in which the source coder output (which is also the channel 
input sequence) contains redundancy. Using this result, we can design a decoder which will 
take advantage of dependence in the channel input sequence. The physical structure of the 
decoder can be easily obtained by examining the quantity to be maximized. The optimum 
decoder maximizes P(Y\Y)P(Y) or equivalently, log P(Y\Y)P(Y), but 

lo g P(Y\Y)P(Y) = X>S P(yi\yi)P(yi\yi-i) (2) 

which is similar in form to the path metric of a convolutional decoder. Error correction 
using convolutional codes is made possible by explicitly limiting the possible codeword to 
codeword transitions, based on the previous code input and the coder structure. At the 
receiver the decoder compares the received data stream to the a priori information about 
the code structure. The output of the decoder is the sequence that is most likely to be 
the transmitted sequence. In the case where there is residual structure in the source coder 
output, the structure makes some sequences more likely to be the transmitted sequence, 
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given a particular received sequence. In other words, even when there is no structure being 
imposed by the encoder, there is sufficient residual structure in the source coder output that 
can be used for error correction. The structure is reflected in the conditional probabilities, 
and can be used via the path metric in (2) in a decoder similar in structure to a convolutional 
decoder. However, to implement this decoder we need to be able to compute the path metric. 

Examining the branch metric, we see that it consists of two terms log P(y f |y,-) and log P(y i \y { _ 1 ) 
The first term depends strictly on our knowledge of the channel. The second term depends 
only on the statistics of the source sequence. Therefore knowledge of both the channel and 
source statistics is necessary for implementing this path metric. In our simulations we have 
obtained the channel statistics by assuming that the channel is a binary symmetric channel 
with known probability of error. We have obtained the second term using a training se- 
quence. The sequence used for testing the proposed approach is different from the training 
sequence. 

In [26] we showed that the use of the decoder led to dramatic improvements under high 
error rate conditions. However at low error rates the performance improvement was from 
nonexistent to minimal. This is in contrast to standard error correcting approaches, in which 
the greatest performance improvements are at low error rates, with a rapid deterioration in 
performance at high error rates. In this work we combine the two approaches to develop a 
joint source channel codec which provides protection equal to the standard channel encoders 
at low error rates while also providing significant error protection at high error rates. 


3 Convolutional Encoders and Joint Source/ Channel 
Decoder 

In [27] the output sequence of the source coder was taken as the sequence { t/,- } . The received 
sequence {y, - } formed the input to the joint source/channel (JSC) decoder which was simply 
a viterbi decoder with a path metric similar to (2). The output of the JSC decoder was 
then passed to the source decoder. As mentioned above, this approach provided significant 
improvements only at high error rates. If we had used a standard convolutional encoder 
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with the source coder, this would have provided excellent error protection at low error rates 
(of course with an increase in the transmission rate). However in spite of the increase in 
transmission rate the convolutional coder still does not provide any protection at high error 
rates 

The convolutional decoder uses the structure imposed by the encoder and the Hamming 
metric to provide error protection. The decoder does not use any of the residual structure 
from the source coder output. We can make use of the residual structure by noting that the 
path labels transmitted by the convolutional encoder comprise the channel input alphabet 
{?/,}. We can then use a training sequence to obtain the transition probabilities { P(y,|y,_i)}, 
and an estimate of the channel error probability to obtain {P(y,|y t )}. These can be used to 
compute the branch metric L 

L = log P(y t - !?/,•_!) + log P(yi\y{) (3) 


which can be used instead of the Hamming metric in the decoder. 

We simulated this approach using a two bit DPCM system as the source encoder. We used 
the two images ‘shown in Figure 1 as the source. The USC Girl image was used for training 
(obtaining the requisite transition probabilities) and the USC Couple image was used for 
testing. The output of the DPCM system was encoded using a (2,1,3) convolutional encoder 
with connection vectors 

g {1) = 64 gM = 74 ( 4 ) 


The convolutional encoder was obtained from [32]. The performance of the different systems 
was evaluated using two different measures. One was the reconstruction signal-to-noise ratio 
(RSNR) defined as 


RSNR — 101og 10 



( 5 ) 


where u,- is the input to the source coder (source image) and « t - is the output of the source 
decoder (reconstructed image). The other performance measure was the decoded error 
probability. The received sequence was decoded using a standard convolutional decoder and 
the JSC decoder. A block diagram of the system is shown in Figure 2. The results are 
presented in Figure 3. Looking at the decoded error performance results in Figure 3a we 


7 



see that the performance seems to be about what we expected from the results of [27]: an 
improvement at high error rates for the system with the JSC decoder with some loss in 
performance at lower error rates. However, from Figure 3b we see that while the decoded 
error probability went down for the system with the JSC decoder at high error rates, there 
was no commensurate improvement in RSNR. In fact for probabilities of error below 0.15, 
there is a decrease in the RSNR for the system with the JSC decoder over a conventional 
system. While we had expected some slight disadvantage for the system with the JSC 
decoder at low error rates, the disadvantage here is not slight and is not restricted to low 
error rates. 

To see why this happened, let us examine our assumptions. The use of the JSC decoder 
was predicated on the assumption that there was some structure in the source coder output. 
The unspoken assumption was that this structure would get translated to the channel coder 
output. In this particular example the source coder output is a sequence of two bit values. 
The channel coder takes the source coder output one bit at a time to generate the two bit 
convolutional coder output. It is highly unlikely that the sample- to-s ample redundancy in 
the source coder output would translate to a bit-to-bit redundancy in the quantizer labels 
which could then be transferred to the convolutional encoder output. Thus our unspoken 
assumption is being violated, and the results in Figure 3 reflect this fact. 

If the above hypothesis is indeed true, then the destruction of the structure in the source 
coder output could be prevented if we used a convolutional coder which uses an input 
wordlength k of two. In order to verify this hypothesis we used a (4,2,1) convolutional 
coder which is equivalent to the (2,1,3) coder in terms’ of rate and memory, but maps the 
two bit outputs from the source coder directly to the channel coder output. The connection 
vectors for this coder are [32] 


rf> = 6 

si 21 

o 

II 

w- 

II 

o> 

ll 

WH 

o 

II 

9 ? 

co 

II 

ll 

II 


The results for the (4, 2, 1) coder, shown in Figure 4, seem to bear out our hypothesis. The 
decoded probability of error is uniformly better for the system with the JSC decoder. The 
same is true for the RSNR with an improvement of about 4 dB at P(e) = 0.1, and about a 
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6 dB improvement at P(e) = 0.25. This is a marked contrast to the results for the (2, 1,3) 
code shown in Figure 3b where there is a 4 dB degradation in the performance of the system 
with the JSC decoder at P(e) = 0.1. 

The two rate 1/2 systems are compared in Figure 5. We have not included the results for the 
(2,1,3) system with the JSC decoder to reduce clutter. Notice that while the conventional 
(2, 1,3) system is superior to the conventional (4,2, 1) system at low error rates, the (4, 2, 1) 
system with the JSC decoder outperforms it as well. 

The simulations were repeated with a rate 2/3 (3,2,2) convolutional coder with connection 
vectors 

flf* = 7 9? = 1 2i 3) = 4 

9? = 2 g? = 5 g { ? = 7 

The results are shown in Figure 6. Notice that while there is some drop in performance for 
the system with the JSC decoder at low error rates, the overall performance is as expected. 
There is an improvement of about 6 dB at P(e) = 0.1. 

In this section we have shown how the use of the residual redundancy in the source coder out- 
put can improve the performance of conventional source coder/convolutional coder systems. 
In order to make use of this redundancy we see that the channel coder input characteristics 
have to match the source coder output characteristics. In the next section we take this 
approach one step further and design channel coders with the specific goal of taking advan- 
tage of the redundancy in the source coder output. An additional advantage of the coders 
described in the next section is that there is an automatic match between the source coder 
output and the channel coder input. 


4 A Modified Convolutional Encoder 

Given that the preservation of the structure in the source coder output requires the channel 
coder input alphabet to have a one-to-one match with the generally nonbinary source coder, 
we propose a general nonbinary convolutional encoder (NCE) whose input alphabet has the 
requisite property. 
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Let x n , the input to the NCE, be selected from the alphabet A — {0, 1, 2 , N - 1}, and let 
y„, the output alphabet of the NCE, be selected from the alphabet S = {0, 1,2, ..., M - 1}. 
Then the proposed NCEs can be described by the following mappings 

1. M = N 2 \ y n = Nx n _x + x n 

The number of bits required to represent the output alphabet using a fixed length code is 

pog 2 (M)l = riog 2 (iV 2 )l = |21og 2 (iV)l 

Therefore in terms of rate, this coder is equivalent to a rate 1/2 convolutional encoder. The 
encoder memory in bits is 2[log 2 (AT)] as each output value depends on two input values. 

As an example, consider the situation when N = 4. Then A = {0, 1,2,3} and S = 
{0,1,2,..., 15}. Given the input sequence x n : 0130211033 and assuming the 
encoder is initialized with zeros, the output sequence will be y n : 0 1 7 12 2 9 5 4 3 15. 

The encoder memory is four bits. Notice that while the encoder output alphabet is of size 
N 2 , at any given instant the encoder can only emit one of N different symbols as should be 
the case for a rate 1/2 convolutional encoder. For example if y n _ x = 0, then y n will take on 
a value from {0, 1,2, ..., (N - 1)}. In general, given a value for y n _ x , y n will take on a value 
from {aN , aN + 1) oN + 2, ..., aN + — 1}, where a = y„_i(modiV). This structure can be 

used by the decoder to provide error protection. The encoder is shown in Figure 7a. 

2- M = N 3 \ y n = A r2 x„_ 2 + + x n 

This encoder is equivalent to a rate 1/3 convolutional encoder with an encoder memory in 
bits of 3flog 2 (Af)]. Given the same input as the previous example, the output alphabet for 
the NCE is 

S= {0,1, 2,. ..,63} 

and the output sequence for the same input sequence is 

y n : 0 1 7 28 50 9 37 20 19 15 

The encoder memory is six bits. In this case even though the encoder output alphabet is of 
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size N 3 , at any instant the encoder can only emit one of N symbols. In general, given a value 
for Vn-uVn will take on a value from {0N,0N+1,...,PN+N-1}, where £ = y n _,(morfA 2 ). 
A block diagram of the encoder is shown in Figure 7b. 

3. M = N 3 

Vn = A 2 a:2n + Nx2n-l + ^2n-2 


The final encoder we consider is equivalent to a rate 2/3 convolutional coder. Notice that 
while the input output relationship looks similar to a rate 1/3 encoder, we generate one 
output for every two inputs. Thus, while the number of bits needed to represent one letter 
from the output alphabet is three times the bits needed to represent a letter from the input 
alphabet, the rate is 2/3 because two input letters are represented by a single output letter. 
This coder could be viewed as a rate 2/3 punctured nonbinary convolutional coder. Again, 
assuming a value of 4 for N, the output alphabet is of size 64, and for the input sequence 
used previously, the output sequence is y n : 0 52 35 22 49 3. 

The encoder memory is again 6 bits. The rate of the encoder can also be inferred from the 
fact that while the encoder output alphabet is of size N 3 , at any instant the encoder can 
transmit one of N 2 (instead of N) symbols. Given a value for y n _ u y n can take on a value 
from the alphabet { 7 N 2 , 7 N 2 + l,..., 7 iV 2 + (N 2 - 1)} where 7 = y n . x {modN). A block 
diagram of the encoder is shown in Figure 7c. 

All of these encoders can be designed for any value of N. Furthermore, their input and 
output alphabets as described above can easily be seen as indices to tables of codewords. 

We will exploit this latter property in the next section for allocating codewords to the NCE 
outputs. 

4.1 Binary Encoding of the NCE Output 

We will make use of the residual structure in the source coder output (which is preserved 
in the NCE output) at the receiver. However, we can also make use of this structure in 
selecting binary codes for the NCE output. An intelligent assignment of binary codes can 
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improve the error correcting performance of the system as can be seen from the following 
example. 

Let N be 2, and let us use the rate 1/2 NCE. In this case if y n = 0, y n+1 cannot be 2 or 3, 

because y n = 0 means x n = 0, and t/ n+1 = 2 or 3 means x n = 1. Thus a decoded sequence 
cannot have 2 or 3 following 0. 

Let us assign fixed length codewords to the NCE outputs as 

0 : 00 , 1 : 01 , 2 : 10 , 3 : 11 


Now suppose the transmitted sequence was the all zero sequence, the metric used was the 
Hamming distance, and the received sequence is 00001000000000 ; that is, there is an error 
in the fifth bit. If the receiver decoded the first four bits as 0,0 then it cannot decode the 
fifth and sixth bits as 2 for the reason noted above. The only two options are decoding them 
as 0 or 1. If we decoded them as 0, we could continue decoding the rest of the sequence 
as 0, 0 ... , and the Hamming distance between the received and decoded sequence would 
be one. If we decoded them as 1, we would have to decode the next set of two bits as 2 
or 3 because 0 cannot follow 1. Decoding as 2 gives the smallest Hamming distance so we 
decode the seventh and eighth bit as 2. This gives a total Hamming distance of two for the 
incorrect path. Thus the receiver will select the correct path (the path with the smallest 
Hamming distance). If the assignment had been chosen as 

0 : 00 ; 1 : 11 ; 2 : 10 ; 3 : 01 

then the Hamming distance for the closest incorrect path would have been three instead of 
two. 

When each allowable sequence is equally likely, there is little reason to prefer one particular 
assignment over others. However, when certain sequences are more likely to occur than 
others, it would be useful to make assignments which increase the ‘distance’ between likely 
sequences. While, for small alphabets it is a simple matter to assign the optimum binary 
codewords by inspection, this becomes computationally impossible for larger alphabets. We 
use a rather simple heuristic which, while not optimal, provides good results. 

The number of M bits codewords that have to be assigned are exactly 2 M . Our strategy is 
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therefore to try to maximize the Hamming distance between codewords that are likely to be 
mistaken for one another. 

First we obtain a partition of the alphabet based on the fact that given a particular value 
for y n - 1 , y n can only take on values from a subset of the full alphabet. To see this, consider 
the rate 1/2 NCE; then the alphabet S can be partitioned into the following sub-alphabets: 

So = (0, 1,2, . . . , Af — 1) 

Si = (/V,iV + l,iV + 2,...,2iV-l) 


S N -i = {N(N -l),N(N -1) + 1,N(N-1) + 2,...,N 2 -l) 

where the encoder will select letters from alphabet Sj at time n if j = y n _ 1 (mod./V). Now 
for each sub-alphabet we have to pick N codewords out of M (= N 2 ) possible choices. We 
first pick the sub-alphabet containing the most likely letter. The letters in the sub-alphabet 
are ordered according to their probability of occurrence. We assign a codeword a from the 
list of available codewords to the most probable symbol. Then, assign the complement of a 
to the next symbol on the list. Therefore the distance between the two most likely symbols 
in the list is K = [log 2 M] bits. We then pick a codeword b from the list which has maximum 
distance from a such that the Hamming distance from a and the Hamming distance from 
the complement of a differ by at most one. We assign it and its complement to the next 
two elements on the list. This process is continued until all letters in the subalphabet have 
a codeword assigned to them. 

As an example, consider the case where N = 4. The partitions are 

50 = (0, 1,2,3) 

51 = (4, 5, 6, 7) 

5 2 = (8,9,10,11) 

5 3 = (12,13,14,15) 
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Assuming that 0 is the most probable symbol, we start- by assigning codewords to the S 0 
sub-alphabet. Suppose 

P{ 0) > P{ 3) > P(l) > P{ 2) 

We first pick a 4 bit codeword for 0 as 0000. The next most probable symbol in this 
sub-alphabet is 3; therefore the codeword for 3 is the complement of the codeword for 0; 
3:1111. The codeword for 1 is at a Hamming distance of two from the codeword for 0 and 
the codeword for 3. The codeword 0011 satisfies this requirement; therefore the codeword 
for 1 is 0011 and the codeword for 2 is 1100. Suppose the next symbol which is close in 
probability to the symbol 0 is 4. We select the sub-alphabet containing that symbol which 
is S\. To the symbol 4 we assign a codeword from the list of unassigned codewords which is 
furthest from the codeword for 0. There are several possibilities for this; we pick 1110. We 
then follow the same procedure for the S\ sub-alphabet. Continuing in this manner we get 
the assignments shown in Table 1. 

4.2 Simulation Results 

The proposed nonbinary convolutional encoders were simulated using the same setup as was 
used in the previous simulations. The binary assignments were made using the statistics of 
the training image which again was the USC Girl image. The test image was once more the 
USC Couple image. The simulation results are presented in Figures 8 and 9. 

In Figure 8 the performance of the rate 1/2 NCE is plotted alongside the results for the 
(2,1,3) coder and the (4,2,1) coder. For both situations we used the system with the JSC 
decoder. Recall that in the previous simulations the (4,2,1) coder with the JSC decoder 
substantially outperformed all other rate 1/2 systems. From the results in Figure 8 we can 
see that the rate 1/2 NCE substantially outperforms the (4,2,1) coder with the JSC decoder. 
Comparing these results against the original results in Figure 3 for the conventional (2,1,3) 
coder we see that the rate 1/2 NCE provides impressive gains: at P(e) = 0.1 the gain of 
the NCE over the conventional (2,1,3) system is about 8 dB while at P(e) = 0.25, the gain 
is about 12 dBl Even more important is the fact that the performance of the rate 1/2 NCE 
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is relatively flat over a very wide range of channel conditions. The RSNR at P(e) = 0.1 is 
less than 1 dB below the value obtained under noiseless channel conditions. Over the entire 
range of channel error probabilities from 0 to 0.25, the RSNR drops slightly more than 4 dB 

The performance of the rate 2/3 NCE, while not as impressive as the rate 1/2 NCE on an 
absolute scale, is still excellent in a relative sense. The performance improvement over the 
conventional (3,2,2) coder at P(e) = 0.1 is again about 8 dB. In fact at that error rate the 
rate 2/3 NCE outperforms the conventional rate 1/2 systems. 

5 Conclusion 


In this paper we have presented two ways of using the structure in the source coder output for 
forward error correction. The first approach simply modifies the decoder in a conventional 
source coder/convolutional coder system to take advantage of the residual redundancy. This 
approach would be especially useful in situations where a conventional system was already in 
place and a change in channel characteristics or transmission requirements required that the 
system be updated . The modifications in this case would be relatively modest and would 
only need to be performed at the decoder. The simulation results show improvements of more 
than 6 dB at high error rates. The second approach involves a new design of the channel 
encoder. The simulation results pertaining to this design show excellent performance over a 
wide range of channel error probabilities, from 0.0 to 0.25. The performance improvements 
range as high as 12 dB at high error rates. This design might be especially useful in the 
codecs designed for the mobile radio communication channel. 

An issue that has not been considered in this paper is the effect of mismatch between the 
actual and assumed channel statistics. Another important area of future research is the 
development of a theoretical measure analogous to dj Tee for use in predicting and classifying 
performance of the coders developed in this paper. 
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Table 1: Codeword Assignments 


Symbol 


Symbol 

(Ms 

0 

0000 

8 

1011 

1 

0011 

9 

0111 

2 

1100 

10 

0100 

3 

mi 

11 

1000 

4 

1110 

12 

0101 

5 

1101 

13 

1001 

6 

0001 

14 

1010 

7 

0010 

15 

0110 



